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ABSTRACT 

Objective: To evaluate the validity of the International 
Classification of Diseases, Tenth Revision (ICD-10) 
code N17x for acute kidney injury (AKI) in elderly 
patients in two settings: at presentation to the 
emergency department and at hospital admission. 
Design: A population-based retrospective validation 
study. 

Setting: Southwestern Ontario, Canada, from 2003 
to 2010. 

Participants: Elderly patients with serum creatinine 
measurements at presentation to the emergency 
department (n=36 049) or hospital admission 
(n=38 566). The baseline serum creatinine 
measurement was a median of 102 and 39 days prior 
to presentation to the emergency department and 
hospital admission, respectively. 
Main outcome measures: Sensitivity, specificity 
and positive and negative predictive values of ICD-10 
diagnostic coding algorithms for AKI using a reference 
standard based on changes in serum creatinine from 
the baseline value. Median changes in serum creatinine 
of patients who were code positive and code negative 
for AKI. 

Results: The sensitivity of the best-performing coding 
algorithm for AKI (defined as a >2-fold increase in serum 
creatinine concentration) was 37.4% (95% CI 32.1% to 
43.1 %) at presentation to the emergency department and 
61.6% (95% CI 57.5% to 65.5%) at hospital admission. 
The specificity was greater than 95% in both settings. In 
patients who were code positive for AKI, the median 
(IQR) increase in serum creatinine from the baseline was 
133 (62 to 288) umol/l at presentation to the emergency 
department and 98 (43 to 200) umol/l at hospital 
admission. In those who were code negative, the 
increase in serum creatinine was 2 (-8 to 14) and 
6 (-4 to 20) umol/l, respectively. 
Conclusions: The presence or absence of ICD-1 0 code 
N17x differentiates two groups of patients with distinct 
changes in serum creatinine at the time of a hospital 
encounter. However, the code underestimates the true 
incidence of AKI due to a limited sensitivity. 



ARTICLE SUMMARY 



Article focus 

■ Validation of administrative database codes is a 
prerequisite to their optimal use in research. 

■ The aim of this study was to describe the validity 
of the International Classification of Diseases, 
Tenth Revision (ICD-10) code N17x for acute 
kidney injury (AKI) compared with a reference 
standard based on changes in serum creatinine. 

Key messages 

■ The ICD-10 code N17x for AKI has a moderate 
sensitivity and high specificity. 

■ The sensitivity of the N17x code improves for 
more severe forms of AKI. 

■ The code was successful in identifying a group 
of patients admitted to hospital with a median 
increase in serum creatinine of 98 umol/l. 

Strengths and limitations of this study 

■ This is the first study to provide information on 
the diagnostic performance of ICD-10 code 
N17x for AKI using laboratory values as the 
reference standard. 

■ It was a large population-based validation study 
that included serum creatinine measurements 
from 12 hospitals. 

■ Future validation studies in younger patients are 
required. 

BACKGROUND 

Healthcare administrative databases can 
provide researchers and policy makers with 
information on a large number of patients in 
an efficient manner. When using these data 
resources for clinical or health services 
research, the validity of the research depends 
upon the accuracy of the diagnostic and pro- 
cedural codes that have been recorded. 1 
However, the accuracy of coding is not guar- 
anteed because administrative databases are 
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not primarily intended for research. 2 Consequendy, 
understanding the validity of administrative codes is a 
prerequisite to their optimal use in the assessment of 
patient outcomes. 

Clinically, acute kidney injury (AKI) is characterised 
by an abrupt decline in the renal function that may 
result in disordered fluid, acid-base and electrolyte 
homeostasis and retention of waste products from nitro- 
gen metabolism, such as creatinine and urea and/or a 
decreased urine output. 3-5 Two systems for defining and 
quantifying the severity of AKI are widely used: the 
Acute Kidney Injury Network (AKIN) classification and 
the Risk-Injury-Failure-Loss-ESRD (RIFLE) criteria. 7 
These staging systems define AKI severity according to 
absolute and relative (percentage) increases in serum 
creatinine, a blood test universally used for indicating 
kidney function. While the incidence of AKI is depend- 
ent on the definition used, it is recognised that this con- 
dition is common, affecting 2-9% of patients at hospital 
admission. 8-11 Moreover, patients who develop AKI have 
both poor short-term and long-term outcomes and their 

• "891 2 20 

care is expensive. 

The purpose of the present study was to evaluate the 
accuracy of the International Classification of Diseases, Tenth 
Revision (ICD-10) code N17x for AKI for applications in 
clinical and health services research, particularly in 
pharmacoepidemiological studies. We compared this 
code against changes in serum creatinine concentration 
in two settings: (1) at presentation to the emergency 
department and (2) at hospital admission. In addition, 
we investigated the effect of baseline chronic kidney 
disease (CKD) status on the diagnostic performance of 
the code in the two settings. Based on the findings of a 
previous validation study on ICD-9 codes, we anticipated 
the sensitivity for ICD-10 code N17x would be low, 

• •* ** 81021 22 

improving with more severe definitions of AKI. 
Moreover, we expected higher sensitivity in patients with 
CKD than those without, as the former typically have 
larger absolute increases in serum creatinine for a given 
amount of AKI. 



METHODS 

Study design and setting 

We conducted a population-based retrospective valid- 
ation study using Ontario's linked healthcare administra- 
tive databases and laboratory data from Southwestern 
Ontario. All residents receive universal access to hospital 
and physician services under a single provincial payer 
system, providing a comprehensive set of health adminis- 
trative data. 

Using a diagnostic test assessment framework, we 
obtained diagnostic performance characteristics (sensitivity, 
specificity, positive-predictive value and negative-predictive 
value) of various diagnostic coding algorithms for ICD-10 
code N17x, which is defined as 'acute renal failure'. We 
used changes in serum creatinine from the baseline 
value as the reference standard (see online supplementary 



table SI for a sample two-by-two table) . Moreover, we com- 
pared the change in serum creatinine between patients 
who were N17x code positive with those who were Nl7x 
code negative. 

Our protocol was approved by the institutional review 
board at Sunnybrook Health Sciences Centre (Toronto, 
Ontario). The relevant datasets are held at the Institute 
for Clinical Evaluative Sciences. The reporting of this 
study follows guidelines set out for studies assessing diag- 
nostic accuracy (see online supplementary table S2). 23 

Data sources 

Patient records from the seven databases were linked 
using encrypted unique identifiers. We identified labora- 
tory measurements, including serum creatinine, using a 
system that keeps patients' electronic medical records 
(Cerner, Kansas City, Missouri, USA). 24 This system con- 
tains inpatient, outpatient and the emergency depart- 
ment laboratory measurements for 12 Southwestern 
Ontario hospitals. For a subpopulation, we also obtained 
previous laboratory measurements from Gamma 
Dynacare, a provider of outpatient laboratory services to 
residents in Southwestern Ontario. We obtained 
inpatient and emergency department patient diagnostic 
information from the Canadian Institute of Health 
Information (CIHI) Discharge Abstract Database (DAD) 
and National Ambulatory Care Reporting System data- 
base (NACRS) , respectively. We obtained information on 
inpatient and outpatient physician services from the 
Ontario Health Insurance Plan database (OHIP). We 
ascertained patients' demographic information from the 
Ontario's Registered Persons Database (RPDB) and pre- 
scribed drug use for patients 65 years of age and older 
from the Ontario Drug Benefit database (ODB). These 
databases have been used extensively to research health 
outcomes and health services. 25-28 

Accrual of elderly patients in Two settings: at presentation 
to the emergency department and at hospital admission 

We developed two separate cohorts. The first cohort 
consisted of patients with a serum creatinine measure- 
ment at presentation to the emergency department 
(Emergency Department Cohort). The second cohort 
consisted of patients with a serum creatinine measure- 
ment at hospital admission (Hospitalised Cohort). All 
patients had a baseline serum creatinine measurement 
(described below). The period of accrual was from 
1 June 2003 to 30 September 2010. 

We restricted cohort entry to patients 66 years of age 
and older to ensure at least 1 year of baseline prescrip- 
tion records for all patients. We excluded the following 
patients from both cohorts: (1) those who had end-stage 
renal disease (defined by the receipt of dialysis in the 
120 days prior to the hospital encounter), (2) those who 
received a kidney or liver transplant in the 5 years prior 
to the hospital encounter, (3) those whose date of 
serum creatinine measurement (from Cerner) and the 
hospital encounter (from CIHI-DAD) did not align (less 
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than 1.4% patients excluded for this reason; see online 
supplementary figure SI for explanation) and (4) those 
without at least one serum creatinine measurement in 
the 7-365 days prior to the hospital encounter to serve 
as the baseline value. In cases where a patient had mul- 
tiple baseline measurements we selected the most recent 
one. Additional selection criteria for each cohort are 
described below and illustrated in online supplementary 
figure SI. 

We excluded patients who did not have a serum cre- 
atinine measurement in the emergency department 
from entry to the Emergency Department Cohort. We 
excluded the following patients from entry to the 
Hospitalised Cohort: (1) those with a hospital admission 
that resulted in a stay greater than 90 days (to ensure we 
had data for the full hospital admission to the time of 
discharge, that is, particularly for patients accrued in the 
second half of the year 2010) and (2) those without a 
serum creatinine measurement either in the emergency 
department prior to hospital admission or during the 
first 2 days of hospital admission. When multiple eligible 
hospital presentations were identified for a given patient 
over the study period, we randomly selected one hospital 
encounter for each cohort. 

Within the two cohorts, we identified patients diag- 
nosed with CKD (assessed by the presence of ICD-10 
code N18x defined as 'chronic kidney disease') in the 
5 years prior to their hospital encounter to evaluate 
potential differential classification of AKI in patients 
with baseline CKD compared with those without. 

The reference standard: serum creatinine-based 
definitions of AKI 

Measuring changes in serum creatinine is the most 
common method of identifying AKI in clinical practice. 
We adapted the reference standard used in this study 
from four widely used serum creatinine-based defini- 
tions of AKI: (1) AKIN Stage 1 or greater: >27 pmol/1 
(0.3 mg/dl) or 50% increase in serum creatinine con- 
centration from the baseline, (2) RIFLE Risk: >1. 5-fold 
increase in serum creatinine concentration from the 
baseline, (3) RIFLE Injury: >2-fold increase in serum 
creatinine concentration from the baseline and 
(4) RIFLE Failure: >3-fold increase in serum creatinine 
concentration from the baseline or a baseline serum cre- 
atinine concentration >354 pmol/1 (4.0 mg/dl) with 
>44 pmol/1 (0.5 mg/dl) increase from the baseline. 6 
A Roche Modular Ion Selective Electrode system (Basel, 
Switzerland) was used to measure serum creatinine. 

For the Emergency Department Cohort, we cate- 
gorised the difference between patients' peak (highest) 
serum creatinine concentration at presentation to the 
emergency department and the baseline serum creatin- 
ine concentration into the serum creatinine-based defi- 
nitions. To be classified as having AKI in the 
Hospitalised Cohort, patients had to have a >27 pmol/1 
(0.3 mg/dl) or 50% increase in serum creatinine con- 
centration from their baseline value at the emergency 



department or during the first 2 days of hospital admis- 
sion. This was done to ensure all patients classified as 
having AKI by the serum creatinine-based definitions 
manifested AKI at hospital admission rather than devel- 
oping the condition de novo during the hospital stay. 
However, there may be a delay from the time of injury 
to when the peak serum creatinine concentration is rea- 
lised. Thus, in those with AKI, we categorised the sever- 
ity as defined by the serum creatinine-based definitions, 
using the peak serum creatinine concentration in 
either the emergency department or in the first 5 days 
of hospital admission. 

ICD-10 coding administrative database algorithms for AKI 

Following a discharge from the hospital, trained coders 
review all charts to record appropriate diagnosis codes 
and their associated attributes. The coders follow the 
Canadian Coding Standards developed by CIHI. 29 
According to CIHI's guidelines, the coders are not per- 
mitted to interpret laboratory measurements, but can 
record a condition based on laboratory findings if the 
physician documents the condition as diagnosed in the 
patient's chart. For ambulatory care records (included in 
CIHI-NACRS), coders are allowed to include up to 10 
diagnoses per visit. The first diagnosis listed is the main 
problem for the patient's visit that required evaluation 
and/or treatment or management as determined by the 
physician at the end of the visit. For hospitalisation 
records (included in CIHI-DAD), coders may record up 
to 25 conditions using ICD-10 diagnostic codes. 
Additionally, they must indicate diagnosis type 'M' for the 
condition that was most responsible for the greatest 
portion of the length of stay or used the greatest amount 
of resources. They may also indicate diagnosis type T' for 
any condition that existed prior to the admission and was 
treated during the hospital stay. 29 

In this study, we tested two unique algorithms to iden- 
tify patients with AKI at presentation to the emergency 
department and three unique algorithms to identify 
patients at hospital admission. Each of these algorithms 
used the ICD-10 code N17x, but varied the possible diag- 
nosis types. In the Emergency Department Cohort, we 
examined the code: (1) as the main problem (referred to 
as 'main diagnosis') or (2) in any of the 10 potential diag- 
nostic fields with any diagnosis type (referred to as 'all 
diagnoses'). In the Hospitalised Cohort, we examined 
the code: (1) as the diagnosis type of 'M' (most respon- 
sible; referred to as the 'most responsible diagnosis'), (2) 
as the diagnosis type of T (pre-admit comorbidity; 
referred to as 'admission diagnosis') or (3) in any one of 
the 25 potential diagnosis fields with any diagnosis type 
(referred to as 'all diagnoses'). 

Statistical analysis 

We used descriptive statistics to summarise demographic 
characteristics, comorbidities, prescription drug claim 
information and baseline laboratory measurements for 
patients in both settings. We calculated sensitivity, 
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specificity, positive-predictive value and negative- 
predictive value for each diagnostic coding algorithm 
(formulas presented in online supplementary table SI). 
We calculated 95% CI for single proportions using the 
Wilson Score method. 30 We performed these calcula- 
tions against the four reference standard definitions. 
We expressed the changes in patient serum creatinine 
concentration from the baseline as medians with IQR 
and we compared the means using the Mann-Whitney 
test. We conducted all analyses using the SAS 
(Statistical Analysis Software) V.9.2 (SAS Institute 
Incorporated, Cary, North Carolina, USA, 2008). 

RESULTS 

We identified a total of 36 049 patients for the Emergency 
Department Cohort and 38 566 patients for the 
Hospitalised Cohort. Baseline characteristics of the two 
cohorts are presented in table 1, including the proportion 
of patients who satisfied different definitions of AKI. For 
example, 294 (0.8%) and 567 (1.5%) patients satisfied the 
RIFLE Injury definition of AKI (defined by >2-fold 
increase in serum creatinine concentration), in each 
cohort, respectively. 

The diagnostic performance of the various coding algo- 
rithms is presented in table 2. For both types of hospital 
encounters, 'all diagnoses' was the best performing 
ICD-10 N17x coding algorithm. At presentation to the 
emergency department, the sensitivity of the ICD-10 code 
for the RIFLE Injury definition of AKI (>2-fold increase in 
serum creatinine concentration from the baseline) was 
37.4% (95% CI 32.1% to 43.1%). 

Sensitivities were higher at hospital admission than at 
presentation to the emergency department for all four 
serum creatinine-based definitions of AKI. For example, at 
hospital admission, the sensitivity of the code for the 
RIFLE Injury definition was 61.6% (95% CI 57.5% to 
65.5%). The sensitivity of the code improved for more 
severe definitions of AKI, peaking at the RIFLE Injury def- 
inition. There was no substantial difference in specificity 
for both types of hospital encounters, with values greater 
than 95% in both settings. The positive-predictive value of 
the code decreased for more severe definitions of AKI, 
with a nadir at the RIFLE Injury definition in both 
settings. 

The absolute change in serum creatinine (the peak 
value — the baseline value) and relative change in serum 
creatinine ((the peak value — the baseline value) /the 
baseline value)) for patients with hospital encounters 
who were positive and negative for the ICD-10 Nl7x 
code are presented in table 3. When considering the 'all 
diagnoses' algorithm, 1.2% of patients at presentation to 
the emergency department and 5.2% of patients at hos- 
pital admission were code positive for AKI. 

The median (IQR) absolute change in serum creatin- 
ine concentration for code positive patients was 133 
(62 to 288) nmol/1 and 98 ( 43 to 200) nmol/1 in each 
setting, respectively. The change for code negative 



patients was 2 (-8 to 14) ^mol/1 and 6 (-4 to 20) ^mol/1 
in each setting, respectively. 

When expressed in relative terms, the median (IQR) 
change for code positive patients was 87 (43 to 204)% 
and 69 (28 to 153)% in each setting, respectively. The 
relative change for code negative patients was 2 (—9 
to 15)% and 7 (—5 to 22)% in each setting, respectively. 
In both settings, the difference in the mean absolute 
and relative change in serum creatinine between code- 
positive and code-negative patients was highly statistically 
significant (p<0.001). 

The diagnostic performance of the 'all diagnoses' algo- 
rithm at hospital admission in patients with and without 
CKD is presented in table 4. The sensitivity of the ICD-10 
code for AKIN Stage 1 or greater, RIFLE Risk and RIFLE 
Injury definitions were higher in patients with CKD than 
those without CKD. For example, the sensitivity of the 
code for the RIFLE Injury definition was 75.6% (95% CI 
60.7% to 86.2%) in patients with CKD and 60.5% (95% 
CI 56.2% to 64.5%) in patients without CKD. 

The code demonstrated the highest sensitivity for the 
RIFLE Risk definition in patients with CKD and the 
RIFLE Failure definition in patients without CKD. 
The specificities of the code were lower in patients with 
CKD than those without CKD for all four definitions. 
For example, the specificity of the code for the RIFLE 
Injury definition was 82.6% (95% CI 80.7% to 84.4%) in 
patients with CKD and 96.2% (95% CI 96.0% to 96.4%) 
in patients without CKD. 

The absolute and relative changes in serum creatinine 
at hospital admission in patients with and without CKD 
who were code positive and code negative are presented 
in table 5. When considering the 'all diagnoses' algo- 
rithm, a total of 18.9% of patients with CKD and 4.6% 
of patients without CKD were code positive for AKI. 

The median (IQR) absolute change in serum creatin- 
ine concentration in patients with CKD who were code 
positive was 108 (48 to 215) |imol/l and in patients 
without CKD who were code positive was 95 (43 to 
197) |omol/l. The difference in the absolute change in 
serum creatinine between patients with and without 
CKD who were AKI code positive was not significantly 
different (p=0.910). The median (IQR) absolute change 
in patients with CKD who were code negative was 16 (—8 
to 51) |j,mol/l and in patients without CKD who were 
code negative was 6 (—4 to 19) (imol/1. 

When expressed in relative terms, the median (IQR) 
change in serum creatinine in patients with CKD who 
were code positive was significantly lower than in patients 
without CKD who were code positive (53 (20 to 104)% vs 
72 (29 to 161)%; p<0.0001). The median (IQR) relative 
change in patients with CKD who were code negative was 9 
(—4 to 26) % and in patients without CKD who were code 
negative was 6 (—5 to 22)%. For both patients with and 
without CKD, the difference in the mean absolute and 
relative changes in serum creatinine between code positive 
and negative patients was highly statistically significant 
(p<0.001). 
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Table 1 Baseline characteristics for patients in the Emergency Department and Hospitalised Cohorts 




Emergency Department 


Hospitalised 




Cohort (n=36049) 


Cohort (n=38566) 


Demographics 






Median age (IQR), years 


—7—7 l~7f\ 00\ 

77 (72-83) 


76 (71-82) 


Women (n (%)) 


h noco /CO A \ 

1 9262 (53.4) 


h c\r\~~tr\ I ac\ a\ 

1 9070 (49.4) 


Income Quintile(n (%)) 






One (lowest) 


-70-70 //-ij 0 \ 

7678 (21 .3) 


0OO~7 /OO 0\ 

8027 (20.8) 


Two 


7306 (20.3) 


77CC /OA ^\ 

7765 (20.1) 


Three (middle) 


7062 (19.6) 


7CC A /HO 0\ 

7654 (19.8) 


Four 


61 10 (16.9) 


n-7ri"7 1 A~7 0\ 

6797 (17.6) 


Five (highest) 


7orn /on o\ 
/o01 (20. o) 


~70 h c /or\ o\ 

/81 b (2U.o) 


Year of Cohort Entry(n (%)) 






2003-2004 


3648 (10.1) 


O^OO 1 A~7 l~ \ 

6733 (17.5) 


2005— 200b 


OO/IO /oo o\ 

8o48 (2o\2) 


nocc /oyi r*\ 
925b (24. (J) 


2007-2008 


H H Of" -i /OO 0\ 

1 1954 (33.2) 


H A OOO /OO ^ \ 

1 1380 (29.5) 


2009— 20 1 0 


h onnn /oo c\ 

1 2oyy (00. o) 


n 1 y / (2y.u) 


Kural location(n (%)) 


5397 (15.0) 


7H or /HO C\ 

7165 (18.6) 


Resident in a long-term care facility(n (%)) 


A A t~ A 1 A f~\\ 

1454 (4.0) 


H OOO /O *1 \ 

1298 (3.4) 


Comorbidities* (n (%)) 






Chronic kidney diseasef 


h r" 00 / >i o\ 

1 526 (4.2) 


A OOO / yl 0\ 

1632 (4.2) 


Diabetes mellitus^: 


0 a e\~? /oo o\ 

8497 (23.6) 


O O ZT O /OO yl \ 

8650 (22.4) 


Peripheral vascular disease 


H H 0"7 /O 0\ 
MO/ (o.2) 


0077 /C yl\ 

20/ / (0-4) 


Coronary artery disease§ 


1 CO/17 /AC ~7\ 

1 bo4/ (4b. /) 


loo44 (4o.y) 


Congestive heart failure 


oo n n /o/i o\ 

8860 (24.6) 


OOO/I /OO OA 

9224 (23.9) 


Stroke/transient ischaemic attack 


i yl Oyl /yl n\ 

1434 (4.0) 


1467 (3.8) 


Chronic liver disease 


OO"? /O 0\ 

837 (2.3) 


A r\~t A / 'o o\ 

1074 (2.8) 


MlnrJinntinn 1 Inn* / n /O/ \\ 

Medication use (n (%)) 






Angiotensin-converting enzyme inhibitor 


A 0"70 H /OO 0\ 

13781 (38.2) 


h /i nrrv /oo r \ 

14859 (38.5) 


Angiotensin-receptor blocker 


CC /i 0 /ho H \ 

b540 (18.1) 


ec h ^ /AC OA 

6514 (Ib.y) 


Potassium sparing diuretic 


OO ,1 O / A O H \ 

3643 (10.1) 


OO >1 O /HO 0\ 

3949 (10.2) 


Non-potassium sparing diuretic 


i COnO /yl C 0\ 

\ boUo (45. -r; 


1 / 1 45 ^44. 5; 


Calcium channel blocker 


1 1 / o5 (Otl.1) 


A OCCO /OO C\ 

1^:553 \od.o) 


p-Adrenergic antagonist 


-\ OCylC ZO"7 OA 

1 3b4b (3/ .9; 


A A ceo /oo n\ 
14bbii ^3o.U; 


btatins 


HC7/1C //IO c\ 

1570b (43. b) 


iccm //io n\ 

1bb02 (43.0) 


NSAIDs (excluding aspirin) 


6520 ( 18.1) 


77CH /OA H\ 

7761 (20.1) 


Anticonvulsants 


0007 iC A \ 

22y/ (b.4) 


2244 (O.o) 


Antidepressants 


OH 0~7 /OC C\ 

9187 (25.5) 


0000 /oo o\ 

8938 (23.2) 


Antipsychotics 


h 000 / r" o\ 

1 883 (5.2) 


H OOO / A A \ 

1692 (4.4) 


Benzodiazepines 


nooc /oc h \ 
yOoo (25.1) 


C\A -i A /Oyl yl \ 

9414 (24.4) 


Antineoplastics 


2217 (6.1) 


2377 (6.2) 


Thyroid hormone 


/-v J -7/— 1 / A ~7 A \ 

6172 (17.1) 


OH J - 0 ia r~ o\ 

6150 (15.9) 


Baseline laboratory measurements^ 






Serum creatinine concentration, umol/L, median (IQR) 


O -1 l~7T~ A A 0\ 

91 (75-1 13) 


OO l~7 1~ A A A \ 

90 (75-1 1 4) 


eGFR ml/min/1.73 m 2 ,** median (IQR) 


61 (46-75) 


OO / A ~~7 ~7~7\ 

62 (47-77) 


ebrn caiGgory(n (%)) 






>bU ml/mm/ 1 . to m 


18382 (51 .0) 


On"7H C /CO "7\ 

20/1b (53./) 


45—59 mi/min/i./o m 


nnyi 0 /oc h \ 

9043 (25.1) 


nni -t /oo yl \ 

901 1 (23.4) 


o(J— 44 mi/min/i./om 


CCOO /-i c c\ 

5622 (15.b) 


CCOO / -i A C\ 

5b33 (14.b) 


H C OO Kirt 1 /rv-iiv-i /H 70m2 

i o— 2y mi/min/i ./on 


OiHE /c "7\ 

<i415 (b./) 


OCO"7 /C C\ 

253/ (b.b) 


<1 5 ml/min/1 .73m 


587 (1 .6) 


669 (1 .7) 


Urine dipstick protein(n (%)) 






negative 


4186 (84.0) 


3252 (81 .4) 


0.3g/l 


415 (8.3) 


409 (10.2) 


1 .Og/I 


296 (5.9) 


257 (6.4) 


>3.0g/l 


87 (1.7) 


79 (2.0) 


Serum sodium concentration, mmol/l, median (IQR) 


139 (137-142) 


139 (137-141) 


Serum potassium concentration, mmol/l, median (IQR) 


4.0 (4.0-5.0) 


4.0 (4.0-5.0) 


Continued 
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Table 1 Continued 





Emergency Department 
Cohort (n=36049) 


Hospitalised 
Cohort (n=38566) 


AKI dGiinitions for all pati6nts(n (%)) 






AKIN StagG 1 or groator 


ooxd. (14.7) 


oo/y (1 7 .0) 


KlrLb HISK 


4/0 (1 .0) 


qqa /n o\ 
oo4 \d..o) 


RIPI P Inii irw 

ruri_c injury 




CC7 M C\ 

DO/ \i -Of 


RIFLE Failure 


527 (1.5) 


920 (2.4) 


AKI definitions for patients with CKDt (n (%)) 






AKIN Stage 1 or greater 


524 (34.3) 


644 (39.5) 


RIFLE Risk 


25 (1.6) 


65 (4.0) 


RIFLE Injury 


12 (0.8) 


41 (2.5) 


RIFLE Failure 


154 (10.1) 


246 (15.1) 



'Comorbidities and medication usage in the 5 and 6 months preceding the hospital encounter were considered, respectively. 
tCKD was assessed by the ICD-10 code N18x, defined as 'chronic kidney disease'. 
^Diabetes mellitus was assessed by the diabetic medication use in the previous 6 months. 

§Coronary artery disease includes the receipt of coronary artery bypass graft surgery, percutaneous coronary intervention and diagnoses 
of angina. 

T]The baseline measurements for serum creatinine were taken at a median (IQR) of 102 (41-204) and 39 (16-128) days prior to the hospital 
encounter for the Emergency Department Cohort and the Hospitalised Cohort, respectively. Baseline urine protein and serum sodium and 
potassium were available for a subset of patients. Emergency Department cohort: A total of 4984, 29 746 and 30 040 patients had a baseline 
urine protein and serum sodium and potassium measurement available in the 7 to 365 days prior to the index date, respectively. Hospitalised 
cohort: A total of 3997, 34 407 and 34 538 patients had a baseline urine protein and serum sodium and potassium measurements available in 
the 7-365 days prior to the index date, respectively 
**eGFR was calculated using the CKD-Epi equation. 

CKD-Epi equation:141xmin((serum creatinine in hitioI/L/88.4)/k, 1)"xmax((serum creatinine in |imol/L788.4)/K, 1)~ 1 2O9 x0.993 Age x1.018 
(if female)x1 .159 (if an African-American) k=0.7 for females and 0.9 for males, a=-0.329 for females and -0.411 for males, min=the 
minimum of Scrfc or 1, max=the maximum of Scrfc or 1. Racial information was not available in our data sources and all patients were 
assumed not to be of a non African-Canadian race. This was a reasonable assumption; as of 2006, African-Canadians represented less than 
7% of the Ontario population. Source: http://www12.statcan.ca/census-recensement/2006/dp-pd/hlt/97-562/index.cfm?Lang=E 
eGFR, estimated glomerular filtration rate. 



I III 



DISCUSSION 

In this population-based retrospective validation study, 
we evaluated the diagnostic performance of ICD-10 
code N17x for AKI. We discovered that the best perform- 
ing coding algorithm both at presentation to the emer- 
gency department and at hospital admission was when 
the code was expressed as 'all diagnoses' (ie, when the 
code was indicated by any diagnosis type). 

At hospital admission, the poorest performing coding 
algorithm was the 'most responsible diagnosis'. A likely 
explanation for this is that AKI frequently presents as a 
complication of other conditions requiring hospital admis- 
sion, such as acute myocardial infarction or sepsis. 5 AKI is 
rarely the primary reason for hospital admission so would 
less likely be coded as the 'most responsible diagnosis'. 

The ICD-10 code demonstrated modest sensitivity for 
the serum creatinine-based definitions of AKI, indicating 
its inability to identify a portion of patients who experi- 
enced a clinically significant increase in serum creatin- 
ine. Of the patients who satisfied the RIFLE Injury 
definition of AKI from their increase in serum creatin- 
ine, 61.6 (95% CI 57.5 to 65.5)% of them were code 
positive AKI when the ICD-10 code was expressed as 'all 
diagnoses'. However, the code demonstrated high speci- 
ficity for all four serum creatinine-based definitions. Of 
the patients who did not satisfy the RIFLE Injury defin- 
ition of AKI, 95.6 (95% CI 95.4 to 95.8)% of them were 
code negative for AKI. These characteristics of the 
ICD-10 AKI code are similar to those described in ICD-9 



code validation studies that used serum creatinine-based 
definitions as the reference standard. 8 10 21 22 The use 
of a serum creatinine-based definition of AKI as the ref- 
erence standard is preferred over a chart review, recog- 
nising the latter have appeared in a number of previous 
validation studies. 31-35 

The ICD-10 code showed improved sensitivity for more 
severe definitions of AKI and was the highest for the 
RIFLE Injury definition. This characteristic of the ICD-10 
code is similar to that of ICD-9 codes for AKI. 21 A greater 
increase in serum creatinine reflects more severe AKI, is 
more likely to be documented in the medical chart and 
thus detected by coders. 

The positive predictive value of the ICD-10 code 
decreased for more severe definitions of AKI and was 
the lowest for the RIFLE Injury definition. This fluctu- 
ation in the positive predictive values can in part be 
attributed to the varying prevalence of the four serum 
creatinine-based definitions. 

At both types of hospital encounters, patients who 
were code positive had a significantiy higher increase in 
serum creatinine than those who were code negative 
(p<0.001). In other words, the code does successfully 
differentiate between two groups of patients with and 
without distinct changes in serum creatinine. 

We also assessed the diagnostic performance of the 
ICD-10 code in subgroups of patients with and without 
CKD prior to the hospital encounter. For all definitions, 
with the exception of RIFLE Failure, the code 
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Table 2 Diagnostic performance characteristics of three different algorithms for ICD-10 code N17x using four different serum 
creatinine-based definitions of AKI as the reference standard 



Diagnostic coding algorithm 



Diagnostic performance characteristics (95% CI) 



Definition 



Emergency Department Cohort 



Hospitalised Cohort 



All diagnoses 



Main diagnosis/most 
responsible diagnosis 



AKIN Stage 1 


Sn=7.2 


(6.6 to 8.0) 


Sn=21.8 


(20.9 to 22.8) 




Sp=99.9 


(99.8 to 99.9) 


Sp=98.4 


(98.2 to 98.5) 




PPV=90.4 


(87.2 to 92.8) 


PPV=74.2 


(72.3 to 76.1) 




MP\/_OC O 

IN" V— OO.c 


^OO.O TO OO.O) 


MP\/_Qc o 
INr V— OO. O 


f&A Q tn d\ 

{o^-.y to oo.O) 


RIFLE Risk 


Sn=30.4 


(26.5 to 34.7) 


Sn=56.4 


(53.2 to 59.7) 




Sp=99.2 


(99.1 to 99.3) 


Sp=96.0 


(95.8 to 96.2) 




PPV=33.8 


(29.5 to 38.4) 


PPV=24.7 


(22.8 to 26.6) 




MP\/_QQ 1 

iNr v-yy. i 


^QQ H tn QQ 0\ 

^yy.u to yy.^-j 


MP\/_QO Q 
INr V— yo.y 


/QQ Q QQ -| \ 

^yo.o to yy. i ) 


RIFLE Injury 


Sn=37.4 


(32.1 to 43.1) 


Sn=61.6 


(57.5 to 65.5) 




Sp=99.1 


(99.0 to 99.2) 


Sp=95.6 


(95.4 to 95.8) 




PPV=25.8 


(21 .9 to 30.2) 


PPV=17.3 


(15.7 to 19.0) 




NPV=99.5 


(99.4 to 99.6) 


NPV=99.4 


(99.3 to 99.5) 


RIFLE Failure 


Sn=30.2 


(26.4 to 34.2) 


Sn=59.1 


(55.9 to 62.3) 




Sp=99.2 


(99.2 to 99.3) 


Sp=96.1 


(95.9 to 96.3) 




PPV=37.3 


(32.9 to 42.0) 


PPV=26.9 


(25.0 to 28.9) 




NPV=99.0 


(98.9 to 99.1) 


NPV=99.0 


(98.9 to 99.1) 


AKIN Stage 1 


Sn=4.1 


(3.6 to 4.6) 


Sn=5.1 


(4.6 to 5.7) 




Sp=100.0 


(99.9 to 100.0) 


Sp=99.9 


(99.8 to 99.9) 




PPV=94.7 


(91 .0 to 97.0) 


PPV=90.7 


(87.4 to 93.2) 




NPV=85.8 


(85.4 to 86.1) 


NPV=82.9 


(82.5 to 83.3) 



Admission diagnosis 



RIFLE Risk 


Sn= 


=21 


4 


(17.9 


to 


25.3) 


Sn= 


=18 


7 


(16 


2 


to 


21 


4) 




Sp= 


=99 


6 


(99.6 


to 


99.7) 


Sp= 


=99 


4 


(99 


3 


to 


99 


5) 




PPV= 


=44 


5 


(38.2 


to 


51.0) 


PPV= 


=42 


5 


(33 


7 


to 


47 


5) 




NPV= 


=99 


0 


(98.9 


to 


99.1) 


NPV= 


=98 


1 


(98 


0 


to 


98 


3) 


RIFLE Injury 


Sn= 


=27 


9 


(23.1 


to 


33.3) 


Sn= 


=22 


8 


(19 


5 


to 


26 


4) 




Sp= 


=99 


6 


(99.6 


to 


99.7) 


Sp= 


=99 


3 


(99 


2 


to 


99 


4) 




PPV= 


=36 


1 


(30.2 


to 


42.6) 


PPV= 


=33 


3 


(28 


7 


to 


38 


1) 




NPV= 


=99 


4 


(99.3 


to 


99.5) 


NPV= 


=98 


9 


(98 


7 


to 


99 


0) 


RIFLE Failure 


Sn= 


=22 


0 


(18.7 


to 


25.7) 


Sn= 


=22 


9 


(20 


3 


to 


25 


8) 




Sp= 


=99 


6 


(99.5 


to 


99.7) 


Sp= 


=99 


5 


(99 


5 


to 


99 


6) 




PPV= 


=51 


1 


(44.6 


to 


57.5) 


PPV= 


=54 


4 


(49 


4 


to 


59 


3) 




NPV= 


=98 


9 


(98.7 


to 


99.0) 


NPV= 


=98 


1 


(98 


0 


to 


98 


3) 


AKIN Stage 1 




n/a 








Sn= 


=15 


8 


(15 


0 


to 


16 


7) 
















Sp= 


=99 


2 


(99 


1 


to 


99 


3) 
















PPV= 


=81 


6 


(79 


4 


to 


83 


6) 
















NPV= 


=84 


5 


(84 


1 


to 


84 


8) 


RIFLE Risk 














Sn= 


=43 


1 


(39 


9 


to 


46 


4) 
















Sp= 


=97 


5 


(97 


3 


to 


97 


6) 
















PPV= 


=28 


5 


(26 


2 


to 


31 


0) 
















NPV= 


=98 


6 


(98 


5 


to 


98 


8) 


RIFLE Injury 














Sn= 


=48 


3 


(44 


2 


to 


52 


4) 
















Sp= 


=97 


2 


(97 


0 


to 


97 


4) 
















PPV= 


=20 


5 


(18 


4 


to 


22 


8) 
















NPV= 


=99 


2 


(99 


1 


to 


99 


3) 


RIFLE Failure 














Sn= 


=47 


4 


(44 


2 


to 


50 


6) 
















Sp= 


=97 


6 


(97 


5 


to 


97 


8) 
















PPV= 


=32 


6 


(30 


2 


to 


35 


2) 
















NPV= 


=98 


7 


(98 


6 


to 


98 


8) 



All values are presented as percentages (%). 

To convert serum creatinine from |imol/l to mg/dl divide by 88.4. 

AKI, acute kidney injury; ICD-10, International Classification of Diseases, tenth revision; NPV, negative predictive value; PPV, positive 
predictive value; Sn, sensitivity; Sp, specificity; n/a, not applicable. 
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Table 3 Change in serum creatinine concentration from the baseline in all patients with and without the ICD-10 N17x code 


for AKI (referred to as code positive and code negative) 












Emergency department cohort 


Hospitalised cohort 








Absolute change 


Relative 




Absolute change 


Relative 


Diagnostic coding 




(umol/l) 


change (%) 




(umol/l) 


change (%)* 


algorithm 


uooe N 


Median (lUK) 




N 


Median (lUH) 




All diagnoses 


+ 426 


133 (62 to 288) 


87 (43 to 204) 


2023 


98 (43 to 200) 


69 C?8 tn 1 5^'l 




35623 


2 (-8 to 14) 


2 (-9 to 15) 


36 543 


6 (-4 to 20) 


f \~0 LU d-d.) 


Main diagnosis/ 


+ 227 


187 (89 to 383) 


128 (62 to 295) 


388 


196 (93 to 396) 


121 (49 to 275) 


most responsible 


35822 


2 (-8 to 14) 


2 (-9 to 16) 


38 178 


7 (-4 to 22) 


7 (-4 to 24) 


diagnosis 














Admission 


+ n/a 






1366 


114 (39 to 187) 


75 (30 to 169) 


diagnosis 








37 230 


6 (-4 to 21) 


7 (-4 to 23) 


Both absolute and relative changes in serum creatinine were significantly different between code positive and code negative patients in both 


types of hospital encounters (all p values < 


0.001) (means presented in box plot; online supplementary figures S2 and 3) 




*((peak serum creatinine — baseline serum creatinine)/baseline serum creatinine)). 








To convert serum creatinine from |jmol/l to mg/dl divide by 88.4. 










AKI, acute kidney injury; CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code 


positive; -, code negative; n/a, not applicable. 











demonstrated higher sensitivity in patients with CKD 
than those without CKD. However, the specificity of the 
code was lower in patients with CKD than those without 
CKD (for all definitions of AKI). The latter finding sug- 
gests a portion of patients with stable CKD are misclassi- 
fied as having AKI at their hospital encounter. For 
example, clinicians may not have access to patients' 



baseline serum creatinine measurements, or may make 
an AKI diagnosis without investigating the baseline mea- 
surements. In such cases, an elevated serum creatinine 
concentration at hospital presentation that is no different 
from the baseline value may still be misdiagnosed as AKI. 

Among patients with CKD, presence or absence of the 
ICD-10 AKI code was also able to differentiate between 



Table 4 Diagnostic performance characteristics of the ICD-10 N17x code in hospitalised patients with and without CKD 
using serum creatinine-based definitions of AKI as the reference standard* 




Diagnostic performance characteristics (95% CI) 




Definition 


Patients with CKD 


Patients without CKD 




Sn=35.6 (32.0 to 39.3) 
Sp=92.0 (90.2 to 93.5) 
PPV=74.4 (69.2 to 78.9) 
NPV=68.7 (66.1 to 71.1) 


Sn=20.4(19.4 to 21 .4) 
Sp=98.6 (98.4 to 98.7) 
PPV=74.2 (72.1 to 76.2) 
NPV=85.9 (85.5 to 86.3) 


RIFLE Risk 


Sn=76.9 (65.4 to 85.5) 
Sp=83.5 (81 .6 to 85.3) 
PPV=16.2 (12.5 to 20.8) 
NPV=98.9 (98.1 to 99.3) 


Sn=54.8 (51 .4 to 58.2) 
Sp=96.5 (96.3 to 96.7) 
PPV=26.2 (24.2 to 28.3) 
NPV=98.9 (98.8 to 99.1) 


RIFLE Injury 


Sn=75.6 (60.7 to 86.2) 
Sp=82.6 (80.7 to 84.4) 
PPV=10.1 (7.2 to 13.9) 
NPV=99.2 (98.6 to 99.6) 


Sn=60.5 (56.2 to 64.5) 
Sp=96.2 (96.0 to 96.4) 
PPV=18.5 (16.8 to 20.5) 
NPV=99.4 (99.3 to 99.5) 


RIFLE Failure 


Sn=48.4 (42.2 to 54.6) 
Sp=86.4 (84.5 to 88.1) 
PPV=38.6 (33.4 to 44.2) 
NPV=90.4 (88.7 to 91 .9) 


Sn=63.1 (59.4 to 66.6) 
Sp=96.4 (96.3 to 96.6) 
PPV=24.8 (22.8 to 26.9) 
NPV=99.3 (99.2 to 99.4) 


All values are presented as percentages (%). 

The ICD-10 N17x coding algorithm considered is all diagnoses. 

AKI, acute kidney injury; CKD, Chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; NPV, negative 
predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity. 
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Table 5 Change in serum creatinine concentration from the baseline in hospitalised patients with and without CKD where 
ICD-10 code N17x did and did not indicate AKI (referred to as code positive and code negative)* 


Patients with CKD 




Patients without CKD 




Absolute change 
(umol/l) 


Relative 
change (%)t 




Absolute change 
(umol/l) 


Relative 
change (%)t 


Code N Median (IQR) 




N 


Median(IQR) 




+ 308 108 (48 to 21 5) 
1324 16 (-8 to 51) 


53 (20 to 104) 
9 (-4 to 26) 


1715 
35 219 


95 (43 to 197) 
6 (-4 to 19) 


72 (29 to 161) 
6 (-5 to 22) 


Code positive and code negative patients were significantly different (all p-values <0.001). 

Relative changes in patients with and without CKD were statistically different (p<0.0001). 

The ICD-10 N17x coding algorithm considered is all diagnoses. 

t((peak serum creatinine — baseline serum creatinine)/baseline serum creatinine)). 

To convert serum creatinine from |jmol/l to mg/dl divide by 88.4. 

CKD, chronic kidney disease; ICD-10, International Classification of Diseases, tenth revision; N, number; +, code positive; - 
n/a, not applicable. 


, code negative; 



two groups of patients with distinct changes in their 
serum creatinine from the baseline value. Although the 
difference in mean absolute change in serum creatinine 
was no different in patients with and without CKD who 
had a positive AKI code, the relative change was lower in 
patients with CKD than those without CKD. This is con- 
sistent with physicians defining AKI more commonly in 
absolute rather than in relative terms. With a given abso- 
lute increase in serum creatinine, the relative increase in 
serum creatinine is lower in patients with CKD than 
those without CKD. 

Our study has several strengths. To our knowledge, it is 
the first study to provide information on the diagnostic 
performance of the ICD-10 code for AKI using serum 
creatinine-based definitions of AKI as the reference 
standard. 8 10 21 22 We assessed different diagnostic coding 
algorithms of the ICD-10 code at both presentation to 
the emergency department and at hospital admission. 
Moreover, we evaluated the diagnostic performance of 
the code in subgroups of patients with and without CKD 
prior to the hospital encounter. We studied patients from 
12 hospitals across Southwestern Ontario with representa- 
tion from both academic and community care centres. 
This helped minimise a selection bias. The large number 
of patients resulted in good precision for the estimates 
provided in the study. 

Our study does have some limitations. We evaluated the 
validity of the ICD-10 code for AKI in patients 66 years of 
age and older. These findings should generalise well to 
elderly patients, a segment of the population at a high 
risk of AKI. 11 36 The findings are also useful for pharma- 
coepidemiological studies using Ontario's healthcare 
administrative databases, where prescription information 
on Ontario residents aged 65 and older is available from 
the universal drug benefit plan. However, future valid- 
ation studies in younger patients are needed. Moreover, 
we did not know the degree to which patients with AKI 
were symptomatic from diminished kidney function or 
the indication that prompted presentation to the emer- 
gency department or hospital admission. 

It is important to acknowledge that for the definitions 
of AKI used in this study, we adapted the serum 



creatinine-based component of the AKIN and RIFLE clas- 
sification systems. The AKIN and RIFLE classification 
systems recommend using both serum creatinine and 
urine output measurements in determining the presence 
and severity of AKI. 6 7 In addition, it is recommended 
that the AKIN classification is applied only after an 
optimal state of hydration is achieved. 6 However, urine 
output measurement and hydration status were not avail- 
able in the data sources used in the study. In reality, the 
accuracy of bedside urine output measurement is notori- 
ously poor outside of intensive care settings with an 
indwelling catheter. Nonetheless, the change in serum 
creatinine is a widely used measure of kidney function in 
clinical settings. Moreover, the serum creatinine-based 
component has been solely used to identify patients with 
AKI using the AKIN and RIFLE classification systems in 
previous studies. 37 38 

The median (IQR) period between the baseline serum 
creatinine measurements and the hospital encounter was 
102 (41-204) days for patients who presented to the 
emergency department and 39 (16-128) days for patients 
admitted to the hospital. While these are reasonable 
baseline measurements, the AKIN and RIFLE classifica- 
tion systems require the change in serum creatinine to 
occur within 48 h and within 7 days, respectively. 6 
Although it is likely that serum creatinine changes 
occurred just prior to the hospital encounter, we cannot 
say this with complete certainty given the absence of avail- 
able measurements during this period. 

Finally, we could not examine the validity of outpatient 
claims for AKI in this study. However, the diagnostic per- 
formance of outpatient claims in our jurisdiction is 
notoriously poor. Nonetheless, emergency department 
and hospital inpatient records hold information on more 
severe forms of AKI, which are of particular interest to 
clinicians, researchers and policymakers. Moreover, we 
recognise that we did not capture those patients who may 
have had severe forms of AKI, but did not present to the 
emergency department or hospital, or those who pre- 
sented, but did not have their serum creatinine mea- 
sured. However, the latter situation is unlikely given that 
serum creatinine measurements are a standard 
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laboratory test for most patients who present to a hospital 
encounter for acute medical care. 

CONCLUSION 

Although the use of healthcare administrative databases 
for clinical or healthcare services research has several 
merits, there are inherent limitations, including the 
accuracy with which certain conditions (ie, AKI) can be 
identified. The sensitivity of ICD-10 code N17x for AKI 
was limited, particularly for less-severe definitions. This 
results in an underestimation of the true incidence of 
the condition. Nonetheless, the presence or absence of 
the ICD-10 code successfully differentiates two groups 
of elderly patients with and without distinct increases in 
serum creatinine from baseline values at the time of a 
hospital encounter. The results from this study guide's 
judicious use of ICD-10 code N17x in future research 
using large healthcare administrative databases. 
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